With the increasing prevalence of digital streaming services and various cloud-based computing solutions, the ability to quickly and accurately transfer large amounts of data between remote devices is a critical task. Sending digital data to a destination system through shared resources of a network, such as the internet, a wide area network (WAN), or local area network (LAN), typically involves the arrangement of data into formatted blocks, known as packets, which may have fixed or variable length. Each data packet typically includes a payload, or body, which has the fundamental user data being delivered to the destination, as well as certain supplemental information used for routing and control purposes, which commonly contained at least partially within a header of the data packet. Broadly speaking, the network, sending systems, and receiving systems may use this supplemental information to ensure proper routing and delivery of the payload to the intended destination.
An often unavoidable consequence of transporting data over a packet switched network in this manner is packet loss, which occurs when one or more data packets fail to properly reach their destination. Packet loss can arise due to a variety of factors, including channel congestion, signal degradation, and other reasons. In order to prevent certain network conditions which cause packet loss to occur, while also efficiently using the available bandwidth in a network channel, a variety of error correction techniques have been developed. Moreover, there are a range of transport protocols which may incorporate tools to handle packet loss, and the particular method used to handle packet loss when it does occur depends on the particular transport protocol used during data transfer. Generally speaking, these transport protocols can be classified under two types, reliable protocols and unreliable protocols, which each present certain tradeoffs, and the particular choice of protocol used in any instance may depend on the nature of the data transfer.
Reliable protocols incorporate guarantees that each data packet is delivered to its destination in sequence, retransmitting dropped packets in the event of packet loss. Reliable protocols are often, but not always, connection-oriented protocols and delivery guarantees are typically accomplished by establishing a backchannel from the recipient back to the sender for a particular communication session, which the recipient may use to send some type of acknowledgement receipts to verify that packets were delivered properly. The sender may use these acknowledgments to guide the retransmission process when it is indicated that data packets failed to properly reach their destination. A prevalent and well-known example of a reliable protocol is Transmission Control Protocol (TCP), which is also connection-oriented. Reliable protocols, such as TCP, are well suited to tasks where accurate transfer of data is a chief concern and some amount of delay can be tolerated in the interests of verifying data packets are delivered properly, such as sending text based emails, digital content downloads, and media streaming services in which audio/video can be buffered at the destination system. Unfortunately, the data verification properties and retransmission of data introduces a comparatively large overhead, rendering many reliable protocols undesirable for time-critical applications, including real-time data transfer, such as live audio and/or video streaming, online video gaming, and internet telephony.
Unreliable protocols, by contrast, generally forgo the type of data delivery verifications for particular packets as described above, and are generally characterized by the fact that they do not guarantee that each packet reaches its destination, nor do they ensure that the packets are delivered in the proper sequence. Unreliable protocols are often, but not always, connectionless, and typically do not establish a fixed channel during any particular communication session. Each data packet may instead be routed independently based on the supplemental information contained in each data packet. A prevalent and well-known example of an unreliable protocol is User Datagram Protocol (UDP), which is also connectionless. Since unreliable protocols like UDP have comparatively reduced overhead by forgoing the reliability properties mentioned above, they are better suited for time sensitive applications where minimizing latency is a chief concern, such as the real-time applications mentioned above.
Since unreliable protocols generally forgo retransmission of data packets, a technique known as forward error correction (FEC) is commonly used to handle packet loss when transporting data using an unreliable service. FEC provides the recipient device with the ability to independently reconstruct lost data without the need for the sender to retransmit source packets which fail to be delivered properly. When forward error correction is used, the original source data is typically redundantly coded at the sender side in the form of FEC packets, which are transmitted to the recipient concurrently with the source packets. In the event of a lost source packet, the recipient device may utilize the redundantly coded data contained in the FEC packets to reconstruct the lost data without having to wait for retransmission.
Among types of loss, unreliable protocols and connections such as WiFi are particularly vulnerable to so-called bursty loss, where connection is lost for periods on the order of 50 milliseconds. Existing methods of forward error correction include Reed-Solomon FEC, in which two Reed-Solomon coded versions of a given source frame are sent as FEC frames within 10 milliseconds of the source frame being sent. Reed-Solomon coding allows for reconstruction of the original source frame at the same resolution. If connection loss takes place over 10 milliseconds or more, however, both the source frame and the FEC frames could be lost. No interruption occurs if either the source or the FEC frames are lost, however, losing both will make reconstruction impossible. Thus, while previous methods can help correct connections lost for very short periods of time, there is a need for error correction that can handle longer bursts of interruption.
Importantly, network conditions often vary over time, causing the maximum bitrate available to a sender over a network channel to vary based on present load on the channel. When a sender system attempts to send data packets at a bitrate that exceeds the current available bandwidth of the channel, it can cause congested conditions which trigger severe packet loss in response. This might be tolerable in less time-sensitive applications involving reliable data transport such as TCP, since retransmission of the lost data is guaranteed; however, this may be unacceptable in many real-time applications and other applications involving unreliable transport, as the packet loss may be to such an extent that the recipient is unable to reconstruct the loss data, causing undesirable consequences such as dropout of the signal. On the other hand, when the maximum available bitrate instead far exceeds the bitrate offered by the sender, this is also undesirable, as the full transmission capabilities of the network channel are inefficiently utilized, and the quality of the signal at the recipient side may be unnecessarily poor as a result. Error correction thus must be able to dynamically adjust to network conditions to ensure effective coverage of packet loss while not creating new packet loss by so many error correction packets that bandwidth capacity is exceeded.
Unfortunately, it is a significant challenge to transfer data using an unreliable protocol in a way that efficiently utilizes the available bandwidth of a network channel without causing congested conditions that result in unacceptable packet loss. Traditional congestion control techniques are often only suitable for reliable protocols, such as TCP, which have feedback to the sender built in to the transport layer, but are ineffective for many unreliable protocols, such as UDP, which typically lack the needed feedback unless independently added over the transport layer by the user. Moreover, while packet loss resulting from increasing the bitrate to the point of congestion might be tolerable in less time-sensitive applications, which use TCP or other reliable protocols to retransmit data, it may be unacceptable in many real-time applications due to a resulting inability of the recipient to reconstruct the data.
Accordingly, there is a need in the art for dynamic forward error correction and effective congestion control and congestion avoidance techniques which are suitable for use with UDP and other unreliable transport protocols that experience periodic and bursty loss. It is within this context that aspects of the present disclosure arise.